Practical Lab 10 - Vanilla CNN and FineTune VGG16 - for Dogs and Cats Classification

Introduction

Lab 10 focuses on the implementation of Convolutional Neural Networks (CNNs) for the classification of dogs and cats. The primary objective of this lab is to explore two different approaches to building and training deep learning models: creating a Vanilla CNN from scratch and fine-tuning a pre-trained VGG16 model for the specific task of classifying images of dogs and cats.

The train folder contains 25,000 images of dogs and cats. Each image in this folder has the label as part of the filename. The test folder contains 12,500 images, named according to a numeric id.

from tensorflow import keras
from tensorflow.keras import layers
import pathlib
from tensorflow.keras.utils import image_dataset_from_directory
import matplotlib.pyplot as plt
import seaborn as sns

# This should point to the small dataset of the Kaggle Dogs vs Cats competition that was created in a previous notebook
data_folder = pathlib.Path('./data/kaggle_dogs_vs_cats_small')

Data Explorarion

train_dataset = image_dataset_from_directory(
    data_folder / "train",
    image_size=(180, 180),
    batch_size=32)
validation_dataset = image_dataset_from_directory(
    data_folder / "validation",
    image_size=(180, 180),
    batch_size=32)
test_dataset = image_dataset_from_directory(
    data_folder / "test",
    image_size=(180, 180),
    batch_size=32)

Found 2000 files belonging to 2 classes.
Found 1000 files belonging to 2 classes.
Found 2000 files belonging to 2 classes.

# Get class names
class_names = train_dataset.class_names

# Count the number of images in each class for training set
train_class_counts = [len(list(train_dataset.as_numpy_iterator())[i][1]) for i in range(len(class_names))]

# Count the number of images in each class for validation set
validation_class_counts = [len(list(validation_dataset.as_numpy_iterator())[i][1]) for i in range(len(class_names))]

# Count the number of images in each class for test set
test_class_counts = [len(list(test_dataset.as_numpy_iterator())[i][1]) for i in range(len(class_names))]

# Plotting the distribution of images for each class in each dataset
plt.figure(figsize=(12, 6))

# Training set
plt.subplot(1, 3, 1)
sns.barplot(x=class_names, y=train_class_counts)
plt.title('Training Set')
plt.xlabel('Class')
plt.ylabel('Count')

# Validation set
plt.subplot(1, 3, 2)
sns.barplot(x=class_names, y=validation_class_counts)
plt.title('Validation Set')
plt.xlabel('Class')
plt.ylabel('Count')

# Test set
plt.subplot(1, 3, 3)
sns.barplot(x=class_names, y=test_class_counts)
plt.title('Test Set')
plt.xlabel('Class')
plt.ylabel('Count')

plt.tight_layout()
plt.show()

It is clear from the graph that the number of photos in each class is equal. The dataset has zero skewness and is class balanced.

type(train_dataset)

tensorflow.python.data.ops.batch_op._BatchDataset

for data_batch, labels_batch in train_dataset:
    print("data batch shape:", data_batch.shape)
    print("labels batch shape:", labels_batch.shape)
    break

data batch shape: (32, 180, 180, 3)
labels batch shape: (32,)

labels_batch

<tf.Tensor: shape=(32,), dtype=int32, numpy=
array([0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1,
       1, 1, 1, 0, 1, 1, 0, 0, 0, 0])>

Each of the 32 photos has RGB channels and a resolution of 180 by 180 pixels. Additionally, the label data is presented in the form 32, where a one-dimensional array containing the labels for each of the 32 photos is provided.

# import imshow
import matplotlib.pyplot as plt

plt.imshow(data_batch[0].numpy().astype("uint8"))

<matplotlib.image.AxesImage at 0x13a5ae3e110>

# Display a few sample images from the training set
plt.figure(figsize=(12, 12))
for images, labels in train_dataset.take(1):
    for i in range(9):
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(images[i].numpy().astype("uint8"))
        plt.title(class_names[labels[i]])
        plt.axis("off") 
plt.show()

Define a Model

Vanilla (CNN)

inputs = keras.Input(shape=(180, 180, 3))
x = layers.Rescaling(1./255)(inputs)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)

model.summary()

Model: "model_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_9 (InputLayer)        [(None, 180, 180, 3)]     0         
                                                                 
 rescaling_1 (Rescaling)     (None, 180, 180, 3)       0         
                                                                 
 conv2d_5 (Conv2D)           (None, 178, 178, 32)      896       
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 89, 89, 32)       0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 87, 87, 64)        18496     
                                                                 
 max_pooling2d_5 (MaxPooling  (None, 43, 43, 64)       0         
 2D)                                                             
                                                                 
 conv2d_7 (Conv2D)           (None, 41, 41, 128)       73856     
                                                                 
 max_pooling2d_6 (MaxPooling  (None, 20, 20, 128)      0         
 2D)                                                             
                                                                 
 conv2d_8 (Conv2D)           (None, 18, 18, 256)       295168    
                                                                 
 max_pooling2d_7 (MaxPooling  (None, 9, 9, 256)        0         
 2D)                                                             
                                                                 
 conv2d_9 (Conv2D)           (None, 7, 7, 256)         590080    
                                                                 
 flatten_3 (Flatten)         (None, 12544)             0         
                                                                 
 dense_4 (Dense)             (None, 1)                 12545     
                                                                 
=================================================================
Total params: 991,041
Trainable params: 991,041
Non-trainable params: 0
_________________________________________________________________

model.compile(loss="binary_crossentropy",
              optimizer="rmsprop",
              metrics=["accuracy"])

callbacks = [
    keras.callbacks.ModelCheckpoint(
        filepath="./models/convnet_from_scratch.keras",
        save_best_only=True,
        monitor="val_loss")
]
history = model.fit(
    train_dataset,
    epochs=30,
    validation_data=validation_dataset,
    callbacks=callbacks)

Epoch 1/30
63/63 [==============================] - 87s 1s/step - loss: 0.6946 - accuracy: 0.5055 - val_loss: 0.6890 - val_accuracy: 0.5200
Epoch 2/30
63/63 [==============================] - 89s 1s/step - loss: 0.6833 - accuracy: 0.5685 - val_loss: 0.6780 - val_accuracy: 0.5630
Epoch 3/30
63/63 [==============================] - 92s 1s/step - loss: 0.6586 - accuracy: 0.6310 - val_loss: 0.6079 - val_accuracy: 0.6600
Epoch 4/30
63/63 [==============================] - 89s 1s/step - loss: 0.6030 - accuracy: 0.6670 - val_loss: 0.6810 - val_accuracy: 0.6310
Epoch 5/30
63/63 [==============================] - 91s 1s/step - loss: 0.5834 - accuracy: 0.6920 - val_loss: 0.6057 - val_accuracy: 0.6880
Epoch 6/30
63/63 [==============================] - 96s 2s/step - loss: 0.5591 - accuracy: 0.7215 - val_loss: 0.6401 - val_accuracy: 0.6760
Epoch 7/30
63/63 [==============================] - 84s 1s/step - loss: 0.5160 - accuracy: 0.7440 - val_loss: 0.5923 - val_accuracy: 0.7160
Epoch 8/30
63/63 [==============================] - 85s 1s/step - loss: 0.4930 - accuracy: 0.7595 - val_loss: 0.6018 - val_accuracy: 0.6800
Epoch 9/30
63/63 [==============================] - 86s 1s/step - loss: 0.4522 - accuracy: 0.7875 - val_loss: 0.6292 - val_accuracy: 0.7210
Epoch 10/30
63/63 [==============================] - 86s 1s/step - loss: 0.4063 - accuracy: 0.8215 - val_loss: 0.6430 - val_accuracy: 0.6970
Epoch 11/30
63/63 [==============================] - 85s 1s/step - loss: 0.3756 - accuracy: 0.8300 - val_loss: 0.6102 - val_accuracy: 0.7100
Epoch 12/30
63/63 [==============================] - 85s 1s/step - loss: 0.3119 - accuracy: 0.8745 - val_loss: 0.7016 - val_accuracy: 0.7040
Epoch 13/30
63/63 [==============================] - 86s 1s/step - loss: 0.2622 - accuracy: 0.8875 - val_loss: 0.9080 - val_accuracy: 0.7410
Epoch 14/30
63/63 [==============================] - 86s 1s/step - loss: 0.2136 - accuracy: 0.9195 - val_loss: 0.8854 - val_accuracy: 0.7250
Epoch 15/30
63/63 [==============================] - 85s 1s/step - loss: 0.1772 - accuracy: 0.9280 - val_loss: 0.9299 - val_accuracy: 0.7370
Epoch 16/30
63/63 [==============================] - 86s 1s/step - loss: 0.1353 - accuracy: 0.9450 - val_loss: 0.9073 - val_accuracy: 0.7360
Epoch 17/30
63/63 [==============================] - 87s 1s/step - loss: 0.1184 - accuracy: 0.9555 - val_loss: 1.0427 - val_accuracy: 0.7210
Epoch 18/30
63/63 [==============================] - 86s 1s/step - loss: 0.0690 - accuracy: 0.9790 - val_loss: 1.4820 - val_accuracy: 0.7190
Epoch 19/30
63/63 [==============================] - 86s 1s/step - loss: 0.0897 - accuracy: 0.9700 - val_loss: 1.4243 - val_accuracy: 0.7280
Epoch 20/30
63/63 [==============================] - 86s 1s/step - loss: 0.0646 - accuracy: 0.9775 - val_loss: 1.3160 - val_accuracy: 0.7380
Epoch 21/30
63/63 [==============================] - 87s 1s/step - loss: 0.0588 - accuracy: 0.9790 - val_loss: 1.7019 - val_accuracy: 0.7180
Epoch 22/30
63/63 [==============================] - 86s 1s/step - loss: 0.0537 - accuracy: 0.9810 - val_loss: 1.5554 - val_accuracy: 0.7430
Epoch 23/30
63/63 [==============================] - 88s 1s/step - loss: 0.0643 - accuracy: 0.9810 - val_loss: 1.7193 - val_accuracy: 0.7230
Epoch 24/30
63/63 [==============================] - 87s 1s/step - loss: 0.0480 - accuracy: 0.9830 - val_loss: 1.6202 - val_accuracy: 0.7440
Epoch 25/30
63/63 [==============================] - 87s 1s/step - loss: 0.0408 - accuracy: 0.9850 - val_loss: 1.7955 - val_accuracy: 0.7480
Epoch 26/30
63/63 [==============================] - 87s 1s/step - loss: 0.0351 - accuracy: 0.9875 - val_loss: 2.0095 - val_accuracy: 0.7290
Epoch 27/30
63/63 [==============================] - 87s 1s/step - loss: 0.0459 - accuracy: 0.9820 - val_loss: 1.7524 - val_accuracy: 0.7530
Epoch 28/30
63/63 [==============================] - 88s 1s/step - loss: 0.0484 - accuracy: 0.9875 - val_loss: 1.8536 - val_accuracy: 0.7260
Epoch 29/30
63/63 [==============================] - 88s 1s/step - loss: 0.0526 - accuracy: 0.9860 - val_loss: 2.1167 - val_accuracy: 0.7050
Epoch 30/30
63/63 [==============================] - 88s 1s/step - loss: 0.0400 - accuracy: 0.9890 - val_loss: 1.9951 - val_accuracy: 0.7270

accuracy = history.history["accuracy"]
val_accuracy = history.history["val_accuracy"]
loss = history.history["loss"]
val_loss = history.history["val_loss"]
epochs = range(1, len(accuracy) + 1)
plt.plot(epochs, accuracy, "bo", label="Training accuracy")
plt.plot(epochs, val_accuracy, "b", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, loss, "bo", label="Training loss")
plt.plot(epochs, val_loss, "b", label="Validation loss")
plt.title("Training and validation loss")
plt.legend()
plt.show()

test_model = keras.models.load_model("./models/convnet_from_scratch.keras")
test_loss, test_acc = test_model.evaluate(test_dataset)
print(f"Test accuracy: {test_acc:.3f}")

63/63 [==============================] - 21s 333ms/step - loss: 0.6228 - accuracy: 0.7065
Test accuracy: 0.706

VGG-16 Network

conv_base = keras.applications.vgg16.VGG16(   
    weights="imagenet",  
    include_top=False,
    input_shape=(180, 180, 3))

conv_base.summary()

Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_10 (InputLayer)       [(None, 180, 180, 3)]     0         
                                                                 
 block1_conv1 (Conv2D)       (None, 180, 180, 64)      1792      
                                                                 
 block1_conv2 (Conv2D)       (None, 180, 180, 64)      36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, 90, 90, 64)        0         
                                                                 
 block2_conv1 (Conv2D)       (None, 90, 90, 128)       73856     
                                                                 
 block2_conv2 (Conv2D)       (None, 90, 90, 128)       147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, 45, 45, 128)       0         
                                                                 
 block3_conv1 (Conv2D)       (None, 45, 45, 256)       295168    
                                                                 
 block3_conv2 (Conv2D)       (None, 45, 45, 256)       590080    
                                                                 
 block3_conv3 (Conv2D)       (None, 45, 45, 256)       590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, 22, 22, 256)       0         
                                                                 
 block4_conv1 (Conv2D)       (None, 22, 22, 512)       1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, 22, 22, 512)       2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, 22, 22, 512)       2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, 11, 11, 512)       0         
                                                                 
 block5_conv1 (Conv2D)       (None, 11, 11, 512)       2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, 11, 11, 512)       2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, 11, 11, 512)       2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, 5, 5, 512)         0         
                                                                 
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________

import numpy as np

def get_features_and_labels(dataset):
    all_features = []
    all_labels = []
    for images, labels in dataset:
        preprocessed_images = keras.applications.vgg16.preprocess_input(images)
        features = conv_base.predict(preprocessed_images)
        all_features.append(features)
        all_labels.append(labels)
    return np.concatenate(all_features), np.concatenate(all_labels)

train_features, train_labels =  get_features_and_labels(train_dataset)
val_features, val_labels =  get_features_and_labels(validation_dataset)
test_features, test_labels =  get_features_and_labels(test_dataset)

1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 2s 2s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 1s 1s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 4s 4s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 5s 5s/step
1/1 [==============================] - 2s 2s/step

train_features.shape

(2000, 5, 5, 512)

conv_base  = keras.applications.vgg16.VGG16(
    weights="imagenet",
    include_top=False)

conv_base.summary()

Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_12 (InputLayer)       [(None, None, None, 3)]   0         
                                                                 
 block1_conv1 (Conv2D)       (None, None, None, 64)    1792      
                                                                 
 block1_conv2 (Conv2D)       (None, None, None, 64)    36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, None, None, 64)    0         
                                                                 
 block2_conv1 (Conv2D)       (None, None, None, 128)   73856     
                                                                 
 block2_conv2 (Conv2D)       (None, None, None, 128)   147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, None, None, 128)   0         
                                                                 
 block3_conv1 (Conv2D)       (None, None, None, 256)   295168    
                                                                 
 block3_conv2 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_conv3 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, None, None, 256)   0         
                                                                 
 block4_conv1 (Conv2D)       (None, None, None, 512)   1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
 block5_conv1 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________

conv_base.trainable = False  
conv_base.summary()

Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_12 (InputLayer)       [(None, None, None, 3)]   0         
                                                                 
 block1_conv1 (Conv2D)       (None, None, None, 64)    1792      
                                                                 
 block1_conv2 (Conv2D)       (None, None, None, 64)    36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, None, None, 64)    0         
                                                                 
 block2_conv1 (Conv2D)       (None, None, None, 128)   73856     
                                                                 
 block2_conv2 (Conv2D)       (None, None, None, 128)   147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, None, None, 128)   0         
                                                                 
 block3_conv1 (Conv2D)       (None, None, None, 256)   295168    
                                                                 
 block3_conv2 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_conv3 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, None, None, 256)   0         
                                                                 
 block4_conv1 (Conv2D)       (None, None, None, 512)   1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
 block5_conv1 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
=================================================================
Total params: 14,714,688
Trainable params: 0
Non-trainable params: 14,714,688
_________________________________________________________________

inputs = keras.Input(shape=(180, 180, 3))             
x = keras.applications.vgg16.preprocess_input(inputs)
x = conv_base(x)
x = layers.Flatten()(x)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)  
model = keras.Model(inputs, outputs)

model.summary()

Model: "model_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_13 (InputLayer)       [(None, 180, 180, 3)]     0         
                                                                 
 tf.__operators__.getitem_4   (None, 180, 180, 3)      0         
 (SlicingOpLambda)                                               
                                                                 
 tf.nn.bias_add_4 (TFOpLambd  (None, 180, 180, 3)      0         
 a)                                                              
                                                                 
 vgg16 (Functional)          (None, None, None, 512)   14714688  
                                                                 
 flatten_5 (Flatten)         (None, 12800)             0         
                                                                 
 dropout_3 (Dropout)         (None, 12800)             0         
                                                                 
 dense_7 (Dense)             (None, 1)                 12801     
                                                                 
=================================================================
Total params: 14,727,489
Trainable params: 12,801
Non-trainable params: 14,714,688
_________________________________________________________________

model.compile(loss="binary_crossentropy",
              optimizer="rmsprop",
              metrics=["accuracy"])

callbacks = [
    keras.callbacks.ModelCheckpoint(
        filepath="./models/feature_extraction_with_data_augmentation.keras",
        save_best_only=True,
        monitor="val_loss")
]
history = model.fit(
    train_dataset,
    epochs=50,
    validation_data=validation_dataset,
    callbacks=callbacks)

Epoch 1/50
63/63 [==============================] - 426s 7s/step - loss: 2.2055 - accuracy: 0.8880 - val_loss: 0.4933 - val_accuracy: 0.9650
Epoch 2/50
63/63 [==============================] - 384s 6s/step - loss: 0.6790 - accuracy: 0.9630 - val_loss: 0.5759 - val_accuracy: 0.9670
Epoch 3/50
63/63 [==============================] - 380s 6s/step - loss: 0.5078 - accuracy: 0.9710 - val_loss: 0.5224 - val_accuracy: 0.9730
Epoch 4/50
63/63 [==============================] - 11043s 178s/step - loss: 0.4067 - accuracy: 0.9760 - val_loss: 0.4398 - val_accuracy: 0.9760
Epoch 5/50
63/63 [==============================] - 406s 6s/step - loss: 0.1424 - accuracy: 0.9895 - val_loss: 0.4805 - val_accuracy: 0.9710
Epoch 6/50
63/63 [==============================] - 450s 7s/step - loss: 0.0998 - accuracy: 0.9915 - val_loss: 0.4686 - val_accuracy: 0.9780
Epoch 7/50
63/63 [==============================] - 389s 6s/step - loss: 0.1993 - accuracy: 0.9865 - val_loss: 0.3937 - val_accuracy: 0.9780
Epoch 8/50
63/63 [==============================] - 418s 7s/step - loss: 0.1331 - accuracy: 0.9915 - val_loss: 0.5992 - val_accuracy: 0.9700
Epoch 9/50
63/63 [==============================] - 462s 7s/step - loss: 0.0889 - accuracy: 0.9925 - val_loss: 0.7949 - val_accuracy: 0.9720
Epoch 10/50
63/63 [==============================] - 461s 7s/step - loss: 0.0809 - accuracy: 0.9950 - val_loss: 0.6201 - val_accuracy: 0.9740
Epoch 11/50
63/63 [==============================] - 479s 8s/step - loss: 0.0815 - accuracy: 0.9910 - val_loss: 0.7712 - val_accuracy: 0.9700
Epoch 12/50
63/63 [==============================] - 396s 6s/step - loss: 0.0595 - accuracy: 0.9945 - val_loss: 0.6089 - val_accuracy: 0.9750
Epoch 13/50
63/63 [==============================] - 404s 6s/step - loss: 0.1665 - accuracy: 0.9920 - val_loss: 0.6828 - val_accuracy: 0.9740
Epoch 14/50
63/63 [==============================] - 391s 6s/step - loss: 0.1029 - accuracy: 0.9930 - val_loss: 0.6095 - val_accuracy: 0.9760
Epoch 15/50
63/63 [==============================] - 390s 6s/step - loss: 0.0463 - accuracy: 0.9970 - val_loss: 0.6621 - val_accuracy: 0.9790
Epoch 16/50
63/63 [==============================] - 384s 6s/step - loss: 0.0295 - accuracy: 0.9970 - val_loss: 0.6113 - val_accuracy: 0.9730
Epoch 17/50
63/63 [==============================] - 402s 6s/step - loss: 0.0508 - accuracy: 0.9945 - val_loss: 0.5506 - val_accuracy: 0.9760
Epoch 18/50
63/63 [==============================] - 395s 6s/step - loss: 0.0669 - accuracy: 0.9955 - val_loss: 0.6373 - val_accuracy: 0.9770
Epoch 19/50
63/63 [==============================] - 399s 6s/step - loss: 0.0427 - accuracy: 0.9975 - val_loss: 0.5529 - val_accuracy: 0.9790
Epoch 20/50
63/63 [==============================] - 380s 6s/step - loss: 0.0745 - accuracy: 0.9960 - val_loss: 0.5455 - val_accuracy: 0.9810
Epoch 21/50
63/63 [==============================] - 390s 6s/step - loss: 0.0436 - accuracy: 0.9965 - val_loss: 0.5889 - val_accuracy: 0.9800
Epoch 22/50
63/63 [==============================] - 381s 6s/step - loss: 0.0296 - accuracy: 0.9965 - val_loss: 0.5155 - val_accuracy: 0.9840
Epoch 23/50
63/63 [==============================] - 381s 6s/step - loss: 0.0707 - accuracy: 0.9955 - val_loss: 0.5215 - val_accuracy: 0.9810
Epoch 24/50
63/63 [==============================] - 391s 6s/step - loss: 0.1348 - accuracy: 0.9950 - val_loss: 0.5417 - val_accuracy: 0.9770
Epoch 25/50
63/63 [==============================] - 381s 6s/step - loss: 0.0387 - accuracy: 0.9970 - val_loss: 0.5628 - val_accuracy: 0.9760
Epoch 26/50
63/63 [==============================] - 383s 6s/step - loss: 0.0858 - accuracy: 0.9960 - val_loss: 0.6647 - val_accuracy: 0.9750
Epoch 27/50
63/63 [==============================] - 387s 6s/step - loss: 0.0210 - accuracy: 0.9985 - val_loss: 0.6077 - val_accuracy: 0.9760
Epoch 28/50
63/63 [==============================] - 447s 7s/step - loss: 0.0829 - accuracy: 0.9955 - val_loss: 0.6733 - val_accuracy: 0.9740
Epoch 29/50
63/63 [==============================] - 457s 7s/step - loss: 0.0516 - accuracy: 0.9975 - val_loss: 0.5613 - val_accuracy: 0.9800
Epoch 30/50
63/63 [==============================] - 455s 7s/step - loss: 0.0184 - accuracy: 0.9975 - val_loss: 0.6848 - val_accuracy: 0.9760
Epoch 31/50
63/63 [==============================] - 424s 7s/step - loss: 0.0179 - accuracy: 0.9980 - val_loss: 0.6306 - val_accuracy: 0.9810
Epoch 32/50
63/63 [==============================] - 391s 6s/step - loss: 0.0250 - accuracy: 0.9990 - val_loss: 0.7114 - val_accuracy: 0.9810
Epoch 33/50
63/63 [==============================] - 394s 6s/step - loss: 0.0608 - accuracy: 0.9975 - val_loss: 0.6756 - val_accuracy: 0.9810
Epoch 34/50
63/63 [==============================] - 388s 6s/step - loss: 0.0170 - accuracy: 0.9980 - val_loss: 0.6504 - val_accuracy: 0.9800
Epoch 35/50
63/63 [==============================] - 385s 6s/step - loss: 0.0378 - accuracy: 0.9980 - val_loss: 0.6800 - val_accuracy: 0.9800
Epoch 36/50
63/63 [==============================] - 381s 6s/step - loss: 0.0419 - accuracy: 0.9975 - val_loss: 0.6503 - val_accuracy: 0.9790
Epoch 37/50
63/63 [==============================] - 390s 6s/step - loss: 0.0426 - accuracy: 0.9970 - val_loss: 0.7521 - val_accuracy: 0.9710
Epoch 38/50
63/63 [==============================] - 414s 7s/step - loss: 0.0621 - accuracy: 0.9965 - val_loss: 0.5665 - val_accuracy: 0.9800
Epoch 39/50
63/63 [==============================] - 432s 7s/step - loss: 0.0570 - accuracy: 0.9965 - val_loss: 0.6782 - val_accuracy: 0.9800
Epoch 40/50
63/63 [==============================] - 405s 6s/step - loss: 0.0195 - accuracy: 0.9990 - val_loss: 0.8321 - val_accuracy: 0.9780
Epoch 41/50
63/63 [==============================] - 420s 7s/step - loss: 0.0354 - accuracy: 0.9985 - val_loss: 0.7707 - val_accuracy: 0.9750
Epoch 42/50
63/63 [==============================] - 373s 6s/step - loss: 0.0277 - accuracy: 0.9975 - val_loss: 0.7157 - val_accuracy: 0.9740
Epoch 43/50
63/63 [==============================] - 383s 6s/step - loss: 0.0806 - accuracy: 0.9975 - val_loss: 0.6797 - val_accuracy: 0.9730
Epoch 44/50
63/63 [==============================] - 396s 6s/step - loss: 0.0089 - accuracy: 0.9980 - val_loss: 0.8602 - val_accuracy: 0.9740
Epoch 45/50
63/63 [==============================] - 414s 7s/step - loss: 0.0146 - accuracy: 0.9985 - val_loss: 0.5870 - val_accuracy: 0.9780
Epoch 46/50
63/63 [==============================] - 413s 7s/step - loss: 0.0216 - accuracy: 0.9980 - val_loss: 0.5596 - val_accuracy: 0.9780
Epoch 47/50
63/63 [==============================] - 414s 7s/step - loss: 0.0112 - accuracy: 0.9975 - val_loss: 0.5452 - val_accuracy: 0.9810
Epoch 48/50
63/63 [==============================] - 408s 7s/step - loss: 0.0205 - accuracy: 0.9980 - val_loss: 0.6291 - val_accuracy: 0.9790
Epoch 49/50
63/63 [==============================] - 390s 6s/step - loss: 0.0479 - accuracy: 0.9970 - val_loss: 0.8266 - val_accuracy: 0.9740
Epoch 50/50
63/63 [==============================] - 484s 8s/step - loss: 0.0427 - accuracy: 0.9980 - val_loss: 0.6946 - val_accuracy: 0.9780

acc = history.history["accuracy"]
val_acc = history.history["val_accuracy"]
loss = history.history["loss"]
val_loss = history.history["val_loss"]
epochs = range(1, len(acc) + 1)
plt.plot(epochs, acc, "bo", label="Training accuracy")
plt.plot(epochs, val_acc, "b", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, loss, "bo", label="Training loss")
plt.plot(epochs, val_loss, "b", label="Validation loss")
plt.title("Training and validation loss")
plt.legend()
plt.show()

test_model = keras.models.load_model(
    "./models/feature_extraction_with_data_augmentation.keras")
test_loss, test_acc = test_model.evaluate(test_dataset)
print(f"Test accuracy: {test_acc:.3f}")

63/63 [==============================] - 357s 6s/step - loss: 0.5238 - accuracy: 0.9755
Test accuracy: 0.975

conv_base.summary()

Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_12 (InputLayer)       [(None, None, None, 3)]   0         
                                                                 
 block1_conv1 (Conv2D)       (None, None, None, 64)    1792      
                                                                 
 block1_conv2 (Conv2D)       (None, None, None, 64)    36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, None, None, 64)    0         
                                                                 
 block2_conv1 (Conv2D)       (None, None, None, 128)   73856     
                                                                 
 block2_conv2 (Conv2D)       (None, None, None, 128)   147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, None, None, 128)   0         
                                                                 
 block3_conv1 (Conv2D)       (None, None, None, 256)   295168    
                                                                 
 block3_conv2 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_conv3 (Conv2D)       (None, None, None, 256)   590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, None, None, 256)   0         
                                                                 
 block4_conv1 (Conv2D)       (None, None, None, 512)   1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
 block5_conv1 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, None, None, 512)   2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, None, None, 512)   0         
                                                                 
=================================================================
Total params: 14,714,688
Trainable params: 0
Non-trainable params: 14,714,688
_________________________________________________________________

conv_base.trainable = True
for layer in conv_base.layers[:-4]:   
    layer.trainable = False

model.summary()

Model: "model_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_13 (InputLayer)       [(None, 180, 180, 3)]     0         
                                                                 
 tf.__operators__.getitem_4   (None, 180, 180, 3)      0         
 (SlicingOpLambda)                                               
                                                                 
 tf.nn.bias_add_4 (TFOpLambd  (None, 180, 180, 3)      0         
 a)                                                              
                                                                 
 vgg16 (Functional)          (None, None, None, 512)   14714688  
                                                                 
 flatten_5 (Flatten)         (None, 12800)             0         
                                                                 
 dropout_3 (Dropout)         (None, 12800)             0         
                                                                 
 dense_7 (Dense)             (None, 1)                 12801     
                                                                 
=================================================================
Total params: 14,727,489
Trainable params: 7,092,225
Non-trainable params: 7,635,264
_________________________________________________________________

model.compile(loss="binary_crossentropy",                       
              optimizer=keras.optimizers.RMSprop(learning_rate=1e-5),
              metrics=["accuracy"])

callbacks = [
    keras.callbacks.ModelCheckpoint(
        filepath="./models/fine_tuning.keras",
        save_best_only=True,
        monitor="val_loss")
]
history = model.fit(
    train_dataset,
    epochs=30,
    validation_data=validation_dataset,
    callbacks=callbacks)

Epoch 1/30
63/63 [==============================] - 529s 8s/step - loss: 0.0154 - accuracy: 0.9975 - val_loss: 0.7075 - val_accuracy: 0.9760
Epoch 2/30
63/63 [==============================] - 477s 8s/step - loss: 3.5167e-06 - accuracy: 1.0000 - val_loss: 0.6797 - val_accuracy: 0.9770
Epoch 3/30
63/63 [==============================] - 455s 7s/step - loss: 0.0326 - accuracy: 0.9990 - val_loss: 0.7000 - val_accuracy: 0.9780
Epoch 4/30
63/63 [==============================] - 473s 8s/step - loss: 6.5243e-05 - accuracy: 1.0000 - val_loss: 0.7031 - val_accuracy: 0.9790
Epoch 5/30
63/63 [==============================] - 462s 7s/step - loss: 0.0449 - accuracy: 0.9975 - val_loss: 0.7292 - val_accuracy: 0.9790
Epoch 6/30
63/63 [==============================] - 476s 8s/step - loss: 0.0058 - accuracy: 0.9985 - val_loss: 0.6799 - val_accuracy: 0.9760
Epoch 7/30
63/63 [==============================] - 459s 7s/step - loss: 0.0231 - accuracy: 0.9990 - val_loss: 0.6990 - val_accuracy: 0.9770
Epoch 8/30
63/63 [==============================] - 525s 8s/step - loss: 0.0013 - accuracy: 0.9995 - val_loss: 0.7923 - val_accuracy: 0.9770
Epoch 9/30
63/63 [==============================] - 564s 9s/step - loss: 0.0185 - accuracy: 0.9990 - val_loss: 0.7504 - val_accuracy: 0.9770
Epoch 10/30
63/63 [==============================] - 600s 10s/step - loss: 0.0113 - accuracy: 0.9990 - val_loss: 0.6641 - val_accuracy: 0.9760
Epoch 11/30
63/63 [==============================] - 657s 10s/step - loss: 0.0506 - accuracy: 0.9980 - val_loss: 0.7442 - val_accuracy: 0.9780
Epoch 12/30
63/63 [==============================] - 677s 11s/step - loss: 0.0035 - accuracy: 0.9990 - val_loss: 0.8127 - val_accuracy: 0.9770
Epoch 13/30
63/63 [==============================] - 653s 10s/step - loss: 0.0052 - accuracy: 0.9990 - val_loss: 0.9327 - val_accuracy: 0.9730
Epoch 14/30
63/63 [==============================] - 639s 10s/step - loss: 0.0128 - accuracy: 0.9990 - val_loss: 0.8690 - val_accuracy: 0.9730
Epoch 15/30
63/63 [==============================] - 639s 10s/step - loss: 0.0107 - accuracy: 0.9990 - val_loss: 0.7524 - val_accuracy: 0.9770
Epoch 16/30
63/63 [==============================] - 621s 10s/step - loss: 5.1681e-11 - accuracy: 1.0000 - val_loss: 0.7524 - val_accuracy: 0.9770
Epoch 17/30
63/63 [==============================] - 640s 10s/step - loss: 2.5516e-06 - accuracy: 1.0000 - val_loss: 0.7571 - val_accuracy: 0.9760
Epoch 18/30
63/63 [==============================] - 625s 10s/step - loss: 9.6780e-05 - accuracy: 1.0000 - val_loss: 1.0240 - val_accuracy: 0.9650
Epoch 19/30
63/63 [==============================] - 666s 11s/step - loss: 6.7167e-04 - accuracy: 0.9995 - val_loss: 0.7633 - val_accuracy: 0.9770
Epoch 20/30
63/63 [==============================] - 638s 10s/step - loss: 0.0048 - accuracy: 0.9995 - val_loss: 0.7725 - val_accuracy: 0.9770
Epoch 21/30
63/63 [==============================] - 631s 10s/step - loss: 0.0393 - accuracy: 0.9960 - val_loss: 0.8830 - val_accuracy: 0.9770
Epoch 22/30
63/63 [==============================] - 603s 10s/step - loss: 0.0041 - accuracy: 0.9990 - val_loss: 1.1081 - val_accuracy: 0.9700
Epoch 23/30
63/63 [==============================] - 586s 9s/step - loss: 0.0021 - accuracy: 0.9995 - val_loss: 0.8432 - val_accuracy: 0.9760
Epoch 24/30
63/63 [==============================] - 555s 9s/step - loss: 0.0029 - accuracy: 0.9995 - val_loss: 1.0457 - val_accuracy: 0.9730
Epoch 25/30
63/63 [==============================] - 477s 8s/step - loss: 0.0177 - accuracy: 0.9995 - val_loss: 0.8025 - val_accuracy: 0.9770
Epoch 26/30
63/63 [==============================] - 592s 9s/step - loss: 0.0064 - accuracy: 0.9990 - val_loss: 0.7880 - val_accuracy: 0.9780
Epoch 27/30
63/63 [==============================] - 547s 9s/step - loss: 0.0378 - accuracy: 0.9975 - val_loss: 0.8616 - val_accuracy: 0.9750
Epoch 28/30
63/63 [==============================] - 486s 8s/step - loss: 5.5244e-07 - accuracy: 1.0000 - val_loss: 0.8637 - val_accuracy: 0.9750
Epoch 29/30
63/63 [==============================] - 609s 10s/step - loss: 0.0107 - accuracy: 0.9995 - val_loss: 0.8446 - val_accuracy: 0.9770
Epoch 30/30
63/63 [==============================] - 616s 10s/step - loss: 0.0150 - accuracy: 0.9990 - val_loss: 0.8486 - val_accuracy: 0.9770

acc = history.history["accuracy"]
val_acc = history.history["val_accuracy"]
loss = history.history["loss"]
val_loss = history.history["val_loss"]
epochs = range(1, len(acc) + 1)
plt.plot(epochs, acc, "bo", label="Training accuracy")
plt.plot(epochs, val_acc, "b", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, loss, "bo", label="Training loss")
plt.plot(epochs, val_loss, "b", label="Validation loss")
plt.title("Training and validation loss")
plt.legend()
plt.show()

model = keras.models.load_model("./models/fine_tuning.keras")
test_loss, test_acc = model.evaluate(test_dataset)
print(f"Test accuracy: {test_acc:.3f}")

63/63 [==============================] - 376s 6s/step - loss: 1.2454 - accuracy: 0.9670
Test accuracy: 0.967

Evaluation of Models

# Load the best weights of the Vanilla CNN model
best_vanilla_cnn_model = keras.models.load_model("./models/convnet_from_scratch.keras")

# Evaluate the Vanilla CNN model on the validation dataset
val_loss_vanilla, val_acc_vanilla = best_vanilla_cnn_model.evaluate(validation_dataset)
print(f"Vanilla CNN Model - Validation Loss: {val_loss_vanilla:.3f}, Validation Accuracy: {val_acc_vanilla:.3f}")

# Load the best weights of the Fine-Tuned VGG16 model
best_fine_tuned_vgg_model = keras.models.load_model("./models/fine_tuning.keras")

# Evaluate the Fine-Tuned VGG16 model on the validation dataset
val_loss_vgg, val_acc_vgg = best_fine_tuned_vgg_model.evaluate(validation_dataset)
print(f"Fine-Tuned VGG16 Model - Validation Loss: {val_loss_vgg:.3f}, Validation Accuracy: {val_acc_vgg:.3f}")

32/32 [==============================] - 11s 304ms/step - loss: 0.5923 - accuracy: 0.7160
Vanilla CNN Model - Validation Loss: 0.592, Validation Accuracy: 0.716
32/32 [==============================] - 139s 4s/step - loss: 0.6641 - accuracy: 0.9760
Fine-Tuned VGG16 Model - Validation Loss: 0.664, Validation Accuracy: 0.976

Vanilla CNN Model Evaluation:

• loss: 0.5923
• accuracy: 0.7160
• Validation Loss: 0.592
• Validation Accuracy: 0.716

Insight:

The Vanilla CNN model achieved a validation loss of approximately 0.592, indicating the average loss on the validation dataset during evaluation. The validation accuracy of around 0.716 means that the model correctly classified about 71.6% of the images in the validation dataset.

Fine-Tuned VGG16 Model Evaluation:

• loss: 0.6641
• accuracy: 0.9760
• Validation Loss: 0.664
• Validation Accuracy: 0.976

Insight:

The Fine-Tuned VGG16 model had a validation loss of approximately 0.664, which is slightly higher than the Vanilla CNN model. However, the validation accuracy of about 0.976 is significantly higher, indicating that the Fine-Tuned VGG16 model performed exceptionally well and correctly classified about 97.6% of the images in the validation dataset.

Confusion Matrix

Confusion Matrix on Validation Dataset

from sklearn.metrics import confusion_matrix
import numpy as np
import seaborn as sns

Confusion Matrix on Training Dataset

# Predictions using the best Vanilla CNN model on training dataset
train_predictions_vanilla = best_vanilla_cnn_model.predict(train_dataset)
train_predictions_vanilla = np.round(train_predictions_vanilla).flatten()

# Get the true labels for the training dataset
true_labels_train = np.concatenate([labels for _, labels in train_dataset], axis=0)

# Generate confusion matrix for Vanilla CNN model on training dataset
conf_matrix_train_vanilla = confusion_matrix(true_labels_train, train_predictions_vanilla)

# Plot confusion matrix for Vanilla CNN model on training dataset
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_train_vanilla, annot=True, fmt="d", cmap="Blues", 
            xticklabels=class_names, yticklabels=class_names)
plt.title("Confusion Matrix - Vanilla CNN Model (Training Dataset)")
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.show()

63/63 [==============================] - 19s 293ms/step

# Predictions using the best Fine-Tuned VGG16 model on training dataset
train_predictions_vgg = best_fine_tuned_vgg_model.predict(train_dataset)
train_predictions_vgg = np.round(train_predictions_vgg).flatten()

# Generate confusion matrix for Fine-Tuned VGG16 model on training dataset
conf_matrix_train_vgg = confusion_matrix(true_labels_train, train_predictions_vgg)

# Plot confusion matrix for Fine-Tuned VGG16 model on training dataset
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_train_vgg, annot=True, fmt="d", cmap="Blues", 
            xticklabels=class_names, yticklabels=class_names)
plt.title("Confusion Matrix - Fine-Tuned VGG16 Model (Training Dataset)")
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.show()

63/63 [==============================] - 318s 5s/step

Classification Report

from sklearn.metrics import classification_report

# Generate classification report for Vanilla CNN model on training dataset
class_report_vanilla = classification_report(true_labels_train, train_predictions_vanilla, target_names=class_names)
print("Classification Report - Vanilla CNN Model (Training Dataset):")
print(class_report_vanilla)

# Generate classification report for Fine-Tuned VGG16 model on training dataset
class_report_vgg = classification_report(true_labels_train, train_predictions_vgg, target_names=class_names)
print("Classification Report - Fine-Tuned VGG16 Model (Training Dataset):")
print(class_report_vgg)

Classification Report - Vanilla CNN Model (Training Dataset):
              precision    recall  f1-score   support

         cat       0.51      0.45      0.48      1000
         dog       0.51      0.57      0.54      1000

    accuracy                           0.51      2000
   macro avg       0.51      0.51      0.51      2000
weighted avg       0.51      0.51      0.51      2000

Classification Report - Fine-Tuned VGG16 Model (Training Dataset):
              precision    recall  f1-score   support

         cat       0.51      0.51      0.51      1000
         dog       0.51      0.51      0.51      1000

    accuracy                           0.51      2000
   macro avg       0.51      0.51      0.51      2000
weighted avg       0.51      0.51      0.51      2000

Classification Report

Vanilla CNN Model:

• Achieved an accuracy of 51% on the training dataset.
• Showed moderate precision, recall, and F1-scores for both 'cat' and 'dog' classes.

Fine-Tuned VGG16 Model:

• Also achieved an accuracy of 51% on the training dataset, similar to the Vanilla CNN model.
• Demonstrated comparable performance metrics to the Vanilla CNN model in terms of precision, recall, and F1-scores for both classes.

from sklearn.metrics import precision_recall_curve
import matplotlib.pyplot as plt

# Compute precision-recall curve for Vanilla CNN model on training dataset
precision_vanilla, recall_vanilla, _ = precision_recall_curve(true_labels_train, train_predictions_vanilla)

# Plot precision-recall curve for Vanilla CNN model
plt.figure(figsize=(6, 5))
plt.plot(recall_vanilla, precision_vanilla, "b-", linewidth=2, label="Vanilla CNN")
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.title("Precision-Recall Curve - Vanilla CNN Model (Training Dataset)")
plt.grid()
plt.legend(loc="lower left")
plt.show()

# Compute precision-recall curve for Fine-Tuned VGG16 model on training dataset
precision_vgg, recall_vgg, _ = precision_recall_curve(true_labels_train, train_predictions_vgg)

# Plot precision-recall curve for Fine-Tuned VGG16 model
plt.figure(figsize=(6, 5))
plt.plot(recall_vgg, precision_vgg, "g-", linewidth=2, label="Fine-Tuned VGG16")
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.title("Precision-Recall Curve - Fine-Tuned VGG16 Model (Training Dataset)")
plt.grid()
plt.legend(loc="lower left")
plt.show()

Examples in which the model failed to predict correctly

# Load the best Vanilla CNN model
best_vanilla_cnn_model = keras.models.load_model("./models/convnet_from_scratch.keras")

# Load the best Fine-Tuned VGG16 model
best_fine_tuned_vgg_model = keras.models.load_model("./models/fine_tuning.keras")

# Load the testing dataset
test_dataset = image_dataset_from_directory(
    data_folder / "test",
    image_size=(180, 180),
    batch_size=32)

# Get class names
class_names = train_dataset.class_names

# Make predictions using the best Vanilla CNN model on the testing dataset
test_predictions_vanilla = best_vanilla_cnn_model.predict(test_dataset)
test_predictions_vanilla = np.round(test_predictions_vanilla).flatten()

# Make predictions using the best Fine-Tuned VGG16 model on the testing dataset
test_predictions_vgg = best_fine_tuned_vgg_model.predict(test_dataset)
test_predictions_vgg = np.round(test_predictions_vgg).flatten()

# Get true labels for the testing dataset
true_labels_test = np.concatenate([labels for _, labels in test_dataset], axis=0)

# Find indices where predictions are incorrect for Vanilla CNN model
incorrect_indices_vanilla = np.where(test_predictions_vanilla != true_labels_test)[0]

# Find indices where predictions are incorrect for Fine-Tuned VGG16 model
incorrect_indices_vgg = np.where(test_predictions_vgg != true_labels_test)[0]

# Display example images where the Vanilla CNN model failed to predict correctly
plt.figure(figsize=(12, 12))
for i, idx in enumerate(incorrect_indices_vanilla[:9]):  # Displaying first 9 incorrect predictions
    for image, label in test_dataset.take(idx + 1):  # Take the dataset up to the specified index
        image = image[0].numpy().astype("uint8")  # Access the first image in the batch
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(image)
        plt.title(f"Predicted: {class_names[int(test_predictions_vanilla[idx])]}, Actual: {class_names[int(label[0])]}")
        plt.axis("off")
        break  # Break the loop after accessing the first image
plt.suptitle("Examples of Incorrect Predictions - Vanilla CNN Model", fontsize=16)
plt.show()

# Display example images where the Fine-Tuned VGG16 model failed to predict correctly
plt.figure(figsize=(12, 12))
for i, idx in enumerate(incorrect_indices_vgg[:9]):  # Displaying first 9 incorrect predictions
    for image, label in test_dataset.take(idx + 1):  # Take the dataset up to the specified index
        image = image[0].numpy().astype("uint8")  # Access the first image in the batch
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(image)
        plt.title(f"Predicted: {class_names[int(test_predictions_vgg[idx])]}, Actual: {class_names[int(label[0])]}")
        plt.axis("off")
        break  # Break the loop after accessing the first image
plt.suptitle("Examples of Incorrect Predictions - Fine-Tuned VGG16 Model", fontsize=16)
plt.show()

Found 2000 files belonging to 2 classes.
63/63 [==============================] - 20s 306ms/step
63/63 [==============================] - 274s 4s/step

Conclusion

Model Performance:

Vanilla CNN Model:

• Validation Loss: 0.592
• Validation Accuracy: 0.716

Fine-Tuned VGG16 Model:

• Validation Loss: 0.664
• Validation Accuracy: 0.976

The Fine-Tuned VGG16 model outperforms the Vanilla CNN model significantly in terms of validation accuracy, achieving an accuracy of 97.6% compared to the CNN model's 71.6%. However, the VGG16 model has a slightly higher validation loss.

Confusion Matrix:

Both models exhibit similar performance in terms of confusion matrices on the training dataset, with similar precision, recall, and F1-scores for both cat and dog classes.

Precision-Recall Curve:

The precision-recall curve for both models shows good performance, with higher precision values at various recall levels, especially for the Fine-Tuned VGG16 model.

Overall, the Fine-Tuned VGG16 model stands out as a robust choice for the Dogs vs. Cats classification task, showcasing the benefits of transfer learning and fine-tuning pre-trained models for image classification tasks.